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Reservoir operations need computational models that can 
attend to both its real time data analytics and multi-objective 
optimization. This is now increasingly necessary due to the 
growing complexities of reservoir’s hydrological structures, 
ever-increasing its operational data, and _ conflicting 
conditions in optimizing the its operations. Past related 
studies have mostly attended to either real time data 
analytics, or multi-objective optimization of reservoir 
operations. This review study, based on systematic literature 
analysis, presents the suitability of Recurrent Learning 
Neural Network (RLNN) and Combine Pareto Multi- 
objective Differential Evolution (CPMDE) algorithms for 
real time data analytics and multi-objective optimization of 
reservoir operations, respectively. It also presents the need 
for a hybrid RLNN-CPMDE, with the use of CPMDE in the 
development of RLNN learning data, for reservoir operation 
optimization in a multi-objective and real time environment. 
This review is necessary as a reference for researchers in 
multi-objective optimization and_ reservoir real time 
operations. The gaps in research reported in this review 
would be areas of further studies in real time multi-objective 
studies in reservoir operation. 
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1. Introduction 


Reservoirs, being formed or modified water bodies, are of great economic importance, and 
largely instrumental to environmental (human and aquatic) safety [1,2]. To this end, reservoir 
purposes, which include power generation, water supply for industrial and domestic uses, flood 
control, recreation, agricultural irrigation, and also disposal of wastes on few occasions [3], must 
be duly optimized for effective performance. Many related studies have been conducted in view 
of optimizing reservoir operations. A three-stage generic framework of release rule for reservoir 
[4,5], dynamic rule curves multipurpose reservoir involving flood control [6,7], water flows 
network modeling system [8,9] real time flood control [9,10] and evolutionary algorithms for 
reservoir operations optimization [11,12] among others, are some of these studies. These studies 
made remarkable contributions, such as the storage at the end of operation which is useful for 
conservation and reservoir operations [5], secured downstream reservoir operation [6], synthetic 
stream flow and reservoir regulation (SSARR) models for irrigation return flows [8]. Many of 
these studies [13-15] are for singular objectives, with few on multi-objective optimization of 
real-time reservoir operations [10,16], using different algorithms, including but not limited to 
recurrent learning neural network (RLNN). 


RLNN is an ANN type of architecture with a self-feedback loop and hidden neurons. Single- 
layer feedforward and multilayer feedforward networks (also known as multilayer perceptron) 
are other types of ANN architecture [17,18]. Fundamentally, ANN, modelled after the 
information processing mechanism of human brain and neurons, is a parallel and massively 
simple processing units from distributed processor. In mimicry of human brain, the knowledge of 
the environment is acquired by a learning process and stores the acquired knowledge using 
interneuron connection strength which is known widely as synaptic weight [17,19]. It is mainly 
partitioned into three layers; namely, input, hidden and output layers as found in Figure 1. 


Hidden 


Fig. 1. Structure of ANNs [17]. 


ANN, unlike stochastic models which are process-driven, is a data-driven model, and thus 
capable of dealing with uncertainties in events, varieties of data, and the frequency and the 
velocity of data update [20]. It also has strong linear and non-linear mapping, environmental 
adaptability and learnability [21]. Different criteria used to design tuned mass damper in high- 
rise building were compared [22]. The study considers the soil-structure interaction effects. In 
another study, shear contribution of fiber reinforced polymer was predicted in reinforced 
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concrete beams that are strengthened [23]. The modeling used multigene genetic programming. 
The study produced competitive results showing the efficacy of the algorithm to beam design. A 
fuzzy inference system was successfully used to evaluate building designing process [24]. Other 
examples of applications of optimization to real life situations are provided [25] with an 
application of NSGA-II and DANN model having successful applications [26]. The study by 
[18] presents the use of ANN to predict the mortal compressive strength. Adaptive neuro-fuzzy 
was implemented to adjust the behavior of training data in a new study [27]. 


RLNN has been applied in diverse fields and its extensive application in reservoir operation 
optimization has received considerable attention [28]. Combine Pareto Multi-objective 
Differential Evolution (CPMDE), on the other hand, is an evolutionary algorithm, and of 
different metaheuristics. Evolutionary algorithms (EAs), generally, are computational methods 
modelled after natural theory of evolution. The algorithms comprises of a two way steps where 
selection follows random variation [29,30]. CPMDE works with specified Pareto methods for 
multi-objective optimization [31]. 


This study, as a preliminary contribution to real time multi-objective optimization solutions for 
dam operations, reviews the suitability of a hybrid algorithm of Recurrent Learning Neural 
Network (RLNN), as suitable ANN architecture, and CPMDE, as a Differential Evolution 
algorithm which can be further developed for real time operations. The next sections of this 
paper review past related studies, specifically the architectures of the neural network used in the 
reservoir operation optimization’s studies. The third section further describes RLNN, as the 
identified most suitable type of ANN architecture, for optimizing real time reservoir operations. 
The fourth section describes CPMDE algorithm for optimizing multi-objective reservoir 
operations. The fifth section, as a conclusion, summarizes the need for the hybridization of 
RLNN and CPMDE algorithm for a multi-objective real time optimization of reservoir 
operations. 


2. Review of past related studies 


2.1. Neural network architecture for reservoir operation optimization 


Studies in computational modelling of reservoir operations, in terms of time-based or discrete 
event system engineering, can be categorized into real and non-real time operations. ANN 
techniques of different architectures have been employed for varying objectives in solving the 
optimization problems of reservoir operations. Multilayer feedforward [4,20,32-39], 
backpropagation [38], recurrent learning neural networks [9,10] and single layer feedforward 
[19,40] are noticeable ANN architectures in this regard. The architecture of these ANN varies 
and invariably determine their suitability and performance for individual use cases. 


Feedforward ANN, which are differentiated by the number of processing layers — single and 
multi, allows information movement in only one direction, without touching the node twice. 
Feedforward neural networks do not have memory; consider only the current input, and do not 
act with the order of time. Therefore, it is generally bad at predicting the next event. 
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Backpropagation architecture, on the other hand, describes a backward movement through the 
neural network in order to find some partial derivatives of the weights’ errors in view of 
subtracting them from the weight. These derivatives are used by gradient descent which 
iteratively minimizes the error function. Thus, continuously train the data to identify the best 
fitness. 


Recurrent learning neural network are suitable for sequential data, such as time-series data, 
video, speech and text in natural language processing. It is an ANN architecture with internal 
memory and thus can remember the important things about the inputs received. It has the ability 
of forming better context of understanding sequence compared to other algorithms because it can 
map input to output in a many-to-many. Recurrent neural network tweaks its weight through 
gradient descent and backpropagation through time (BPTT). Table 1 presents a review articles — 
systematically sought, sorted and collated —on optimization of reservoir operations for different 
tasks, using ANN. The review categorizes the studies into real-time and non-real-time 
optimization to highlight the type of ANN architecture employed, the reservoir operation 
designed for, and any supporting algorithm where applicable. 


It is shown, from Table 1, that there are limited available studies on real time framework for 
reservoir operation optimization, when compared with non-real time studies. Bridging this 
scholarly gap is essential. More importantly, considering the evolving technologies for capturing 
reservoir data in real time, such as satellite altimeter and water level sensor, there is ease of 
decision making when such data is readily available for analysis and process optimization. Even 
though multilayer feed-forward network is mostly used in these studies, recurrent neural network 
is the network architecture fundamentally meant for real time optimization and performed better 
than multilayer feedforward network for real time optimization of reservoir operation [10]. 


3. Recurrent learning neural network for optimizing real time reservoir 
operations 


RRLNN is an ANN algorithm built on the recurrent network architecture. Its synaptic weights 
are adjusted in real time, that is, as the network processes its signal. It has input-feedback layer 
with another processing layer comprising of computation nodes. The synaptic connections are 
feedforward and feedback structures. The description of its state-space is given in equations | 
and 2. 


X(n+1)= p(Wax(n) + WbU(n)) (1) 
Y(n) = CX(n) (2) 


Wa is a q-by-q matrix; Wb is a g-by-(m+1) matrix; C is a p-by-q matrix; and @ is the 
activation function, defined as amplitude limiting object of the output. Equation 1 and 2 are the 
process and measurement equations of the RRLNN model. In these, X(7) is the vector state of a 
nonlinear discrete-time system, U(n) is the input applied to the system, and Y(n) is the output of 
the system. 
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Review of ANN-based Studies on Real-Time Reservoir and Non-Real-Time Operation Optimization. 


Supporting algorithm (if 


Reservoir operation | Network Architecture ; Reference 
applicable) 
Forecasting water level | Multilayer feedforward Feed Forward Backpropagation [39] 
é : “Genetic algorithm (GA) and a 
Infelligearcontol | weifndieated fuzzy rule base (FRB) [28] 
system for reservoir ciety 
operation” programme 
P Multilayer feedforward Genetic algorithm [33] 
Prediction model for Multilayer feedforward iuitvatiable Linear Resession [41] 
sediment inflow network 
Real Sere : 
firme Fuzzy rules applied in “Adaptive 
Pareavine Hocd “Real-time recurrent network” | Network-based Fuzzy Inference [10] 
acs ena System (ANFIS)” 
“Real-time recurrent network” | Genetic algorithm and Fuzzy rule [9] 
: “Feedforward algorithm using 
Rainfall forecasting Mulder teccronward hyperbolic tangent transfer [32] 
network as 
function 
Hydropower energy Single step evolving ANN Particle Swarm Optimization [42] 
Hydropower . 
perfomance mediction Single-layer perceptron None [19] 
Predicting sediment Mualilayerpercepuda Radial basis function in dynamic [38] 
estimation programming 
Forecasting reservoir Multilayer feedforward Nees [31] 
storage for network 
Hydropower dam Multilayer feedforward Quantum-behaved particle swarm [21] 
operation network optimization (QPSO) 
Developing reservoir Multilayer feedforward Implicit stochastic optimization [34] 
hedging/operating Multilayer feedforward Dynamic programming [20] 
rules Multilayer feedforward Fuzzy rule [4] 
“Feed-forward neural network 
Multilayer feedforward and error Back-propagation [43] 
learning algorithm”. 
E en ee eh Linear and non-linear regression 
Ne ES nga s Multilayer neural network formulated through dynamic [44] 
flow discharge ; 
Re programming. 
time Multilayer neural network enieue 2 leo inna adeupport [21] 
: y Vector Machine 
Multi-Layer Perceptron None [45] 
: : Multilayer neural network Fuzzy rule [35] 
Forecasting reservoir : 
Multilayer feed-forward F 
water level None [31] 
networks 
Levenberg-Marquardt Back 
Reservoir inflow Propagation (LMBP) None [42] 
algorithm 
Not indicated None [46] 
Simulating Single layer feed-forward youre stoohasnc moder wat 
hydrological aaa vee Thomas-Fiering model, and [46] 
y Hidden Markov Model (HMM)” 
PerOHTEneS Backpropagation algorithm in a 
activities Multilayer feed-forward ie ae 
Hooke and Jeeves nonlinear [36] 
networks : 
programming model. 
Water distribution Multilayer feed-forward Discrete wavelet transform [37] 


prediction 


networks 


(DWT) for noise cleansing 
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Fig. 2. Real time Recurrent Learning Neural Network [47]. 


In summary, the parameters for the real time recurrent learning algorithm are dimensionality of 
input space (m), dimensionality of state space (q), dimensionality of output space (p), and 
synaptic weight vector of neuron (nodes) (Wj), where j = 1, 2,...,q. RRLNN is initialized by (1) 
setting the synaptic weights of the algorithm to selected small values from a uniformly- 
distributed range, (ii) setting the initial value of the state vector X(0) = 0, and (iii) setting 
Aj (0) = 0, (Aj (n) is defined as a partial derivative of the state vector X(n) when the weight 
vector is Wj), for j = 1, 2,...,q. 


The computation is for n = 0,1, 2, ..., where: 


: ox 
Aj) = Sy 3) 


And, Uj (n) is a q-by-(q +m-+1) matrix whose rows are all zeros, except the jth row, and 
@(n) is a g-by-q diagonal matrix, with kth diagonal element being a partial derivative of the 
activation function (@). 


Therefore, the computation to initialize Aj (0) is: 


Aj (n+ 1) = O(n)[Wa(n)Aj (n) + Uj (n)] (4) 
e(n) = d(n) — CX(n) (5) 
AWj (n) = nCAj (n)e(n) (6) 


The core strength of RRLNN in its optimization of real time operations is the use of 
instantaneous gradient (denoted as AWée(n)) of the synaptic weight vector AWj (see eqn. 6), as 
against true gradient (AW¢etotal ) used in non-real time optimization. 


¢(n), the instantaneous sum of squared errors at time n, is defined in terms of e(n) (see equation 
5): 


a(n) = Fe" (n)e(n) (7) 


On another hand, for system stability during real time operation optimization, the learning 
parameter, n (see equation 6), could be made small enough that it makes time scale of the weight 
changes to be smaller than the time scale needed for the network operation. RRLNN, though 
being a computational model capable and sufficient for real time operation optimization, would 


58 AL. Ajala et al./ Journal of Soft Computing in Civil Engineering 4-3 (2020) 52-64 


necessarily be supported by an evolutionary algorithm for a multi-objective operational 
framework. 


4. Combine pareto differential evolution algorithm for optimizing multi- 
objective reservoir operations 


CPMDE can be mathematically represented thus (see eqn.8). 
x[t +1] = s(v(x[t])) (8) 


The population at time t is denoted as x[t], and operated on by random variation v. With the 
selection s, a new population x[t + 1] is produced. 


EAs conceptually simulate evolution of individual structures through selection, mutation and 
crossover processes. They maintain rule-based evolving population structures, with cross over 
and mutations as genetic operators [29]. EAs’ application are recorded in a range of domain 
problems, and the target function is optimized [48,49]. Implementation simplicity, flexibility and 
robustness, and its no-need-for-derivative-information are the factors considered that informed 
the EA usage in some computational optimizations [33,50]. Most importantly, the strength of 
EAs when using it to solve some multi-objective optimization problems is responsible for its 
wide range of domain applicability. 


In solving real world engineering problems, there are always several conflicting criteria and 
goals, and this characterizes multi-objective optimization. EAs are, however, meant to proffer 
adequate solutions for such conflicting multi-objective paradigms through optimization of the 
several conflicting criteria by searching for an optimal vector, and this is not used as a single 
value. The Pareto front is optimal solutions set; a non-dominated and non-inferior set of 
solutions that present as solutions to multi-objective optimization problems [31]. The multi- 
objective optimization problem, according to [50] in its general is presented in Figure 3: 


1 Minimize/Maximize f,, (x), m= 1,2,..,M; 
subject to 
gj(x) 2 0, 5 ee ee fe 
h(x) = 0, a1 2ahk: 
x) <x< ree i=1,2,...,n; 


Fig. 3. Multi-objective Optimization Problem [50]. 


The solution is a variable vector with a corresponding objective vector. It is x € Rn of n 
(L) (U) 
U U 

lower and upper bounds for the variable x;, respectively. Therefore, the solutions to satisfy the 
constraints (2) — (4), as shown above, is a possible decision variable space S C Rn. The 
objective functions (1) constitute a dimensional space M, also called objective space Z C RM. 
There is a point z € RM in the objective space for each solution x in the decision variable 
space. This is denoted by f (x) = z = (2,Zp,...,Zy)’. Multiple Pareto-optimal solutions, in 
solving multi-objective optimization, are therefore meant to discover values close to the Pareto 


decision variables, where x = (X1,X2,-,Xn)’. The values x’ and x,’ are corresponding 
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front as much as possible. It also needs to be as diverse as it can be in the non-dominated front 
obtained [50,51]. 


On another hand, in solving multi-objective optimization problems, DE is a simple but strong 
EA, which has been widely employed. Though DE was initially employed for single-objective 
optimization problems, it has now been successfully applied to multi-objective optimization 
problems [51-55]. The commonality in these DE approaches, however, is that new candidate 
solutions are created by combining the parent individual with other individuals within the same 
population. The new candidate only replaces the parent if it has better fitness, otherwise, it is 
discarded. It is an ambitious selection scheme that outperforms traditional EAs. 


1 “Evaluate the initial population P of random individuals”. 

2 “While stopping criterion not met, do”: 

2.1 “For each individual P;(i = 1,..., popSize) from P repeat”: 
(a) “Create candidate C from P,”. 

(b) “Evaluate the candidate”. 


(c) “If the candidate is better than the parent, the candidate replaces the parent. Otherwise, the 
candidate is discarded”. 
2.2 “Randomly enumerate the individuals in P”. 


Fig. 4. Outline of Differential Evolution Procedure [51]. 


CPMDE is a differential evolutionary algorithm and it works according to the generic procedural 
format of DE (as presented in Figure 4), to solve multi-objective optimization problem (as 
presented in Figure 2). It potentially outperforms traditional EAs, as typical of DE. Its novel 
workability in creation of candidate (see 2.1(a) in Figure 4), or new selection from each 
population generation, is by combining methods of Pareto ranking and Pareto dominance. These 
methods —Pareto ranking and Pareto dominance— are mostly singly used by past related studies. 
In Pareto ranking, ranks are assigned to every solution in both the trial and target population 
based on their non-domination levels. In doing this, solutions are selected with the best ranked 
for propagation to the next generation. In Pareto dominance, however, ranks are not assigned, but 
a winning solution at the dominance space moves to the next generation. CPMDE has been 
found to perfectly handle optimization constraints, provide diverse solution sets, and robust 
enough on tuneable problems [31]. The algorithm procedure of CPMDE is presented in Figure 5. 


“Input the DE parameters required”. 
“Randomly initialize all solution vectors within the limits of the variable bounds”. 


1 

2 

3 “Set the generation counter, g = 0.” 

4 “Generate a trial population using DE’s mutation and crossover operations”. 
5 


“Domination check is performed on the combined trial and target population. All non-dominated 
solutions as ‘non-dominated’ while marking others as ‘dominated’”. 
“Play domination tournaments at each population index”. 


6 

7 “Increase the generation counter, g, by 1, that is,g =g+1”. 
8 “If g < gMax, then go to step (4) above. Else, go to step (9)”. 
9 


“Remove the dominated solutions in the last generation”. 


10 “Output the non-dominated solutions” 


Fig. 5. Combine Pareto Differential Evolution (CPMDE) Algorithm Procedure [31]. 
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5. Conclusion 


Reservoir operations include, but not limited to, hydropower generation, marsh land restoration, 
river sediment estimation, flood and water inflow level forecasting, reservoir control system and 
recreation and conservation, environmental conservation, flood mitigation, and rural-urban water 
supply. These operations differ in mechanism and therefore in the design of their underlying 
hydrological and computational models. Reservoirs designed for many objectives is called multi- 
objective reservoir. It can also be designed for many operations. Therefore, it logically demands 
more complex design models and larger operational data to handle. 


Also, real time reservoir operations are operations whose variables are captured and processed in 
real time using technology for accurate and timely decision making. Instead of the technological, 
but periodic, inspection of reservoirs to capture needed information for decision-making and 
operational optimization, information and communication technologies (ICT), such as 
Piezometers, are permanently located at the reservoirs’ locations. These ICT tools capture, 
transfer and process the needed reservoirs’ operational data for process optimization. Therefore, 
real time reservoir operations’ data are captured, transferred, and processed in real time for 
optimization and decision making. 


The need for RRLNN and CPMDE for multi-objective real time reservoir operation optimization 
can be summarily classified into four. First, with the preponderance of real time data capturing 
technologies, such as Piezometers, satellite altimeter and water level sensor, more data are 
generated real time and needed to be deployed into analytics system, also in real time. This is a 
practical implication of such computational models for domain experts’ decision-making process. 
Second, for specificity, RRLNN and CPMDE are combinable algorithms with prospect to attend 
to the real time and multi-objective functions and constraints of reservoir operations. RRLNN, 
being an extension of recurrent network —an ANN network architecture suitable for real time 
functionality, while CPMDE is a novel DE algorithm suitable for multi-objective optimization. 


Training the RRLNN operational data with CPMDE, with its learning parameter, n, will present a 
computational model that adequately handles the reservoir operational data in real time within a 
multi-objective framework. Third, aside the practical problems in optimizing real time reservoir 
operations within a multi-objective framework that the hybridization of RRLNN and CPMDE 
would solve, the model presents a new frontier in computational modelling. It presents a 
hybridization process whereby the non-dominated vectors from the CPMDE serves the data 
population or the training of the RRLNN. This includes training the RRLNN’s nodes and 
abstracting its data hidden layers. Fourth, the hybrid of RRLNN and CPMDE would be the basis 
for the design and development of an intelligent system that handles multi-objective decision 
making of reservoir operations in real time. 
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